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ABSTRACT 

To verify assumptions concerning content validity 
and to qualify relationships between tests and criteria, three 
validation studies of the United States Employment Service Dictation, 
Spelling, and Typing Tests were carried out in cooperation with 18 
State Employment Services. Study I, validation of the Spelling and 
Dictation Tests, revealed a moderately low positive relationship 
between workers* Dictation Test scores and supervisors' ratings and 
also some relationship between Spelling Test scores and supervisors* 
ratings of ability to take and transcribe dictation. Study II, a 
follow-up on the Typing and Spelling Proficiency Tests, revealed a 
moderate but highly significant correlation between typing scores and 
spelling test scores as well as correlations between typing scores 
and supervisory ratings. Study III, relationship of Typing Test 
performance with supervisors* evaluation of on-the-job performance, 
revealed little or no predictive validity between typing scores and 
supervisors* evaluations and that a satisfactory- unsatisfactory 
criterion was not suitable for this type of research. Summary, 
purpose, procedures, and recommendations are included for each study. 
Selected study forms are appended. (SB) 
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FOREWORD 



Extensive research conducted under the Federal«State cooperative test 
research program in the Training and Employment Service has led to the 
development of many tools useful in vocational counseling and placement. 
These tools include aptitude tests, proficiency tests, and non-cognitive 
measures based on instruments such as interest inventories and biographi- 
ccl information blanks. 

The purpose of this series of reports is to provide results of significant 
test research projects as they are completed. These reports will be of 
interest to users of the tests and to test research personnel in other 
organizations. 

This report was prepared in the Division of Counseling and Testing Services 
of the United States Training and Employment Service by Eileen D. Haggerty 
and Marlin L. Ferral under the general direction of Anthony J. Fantaci, 
Chief of the Division, and Beatrice J. Dvorak, Assistant Chief of the 
Division. Statistical services were provided by John A. Hawk of the 
Division of Counseling and Testing Services. 
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Introduction 



This report describes the results of three validation studies of the 
USES Dictation, Spelling and Typing Tests. Studies of the criterion- 
related validity of proficiency tests are rare because such tests are 
generally acknowledged to possess content validity when the knowledges 
and skills needed for success on the tests correspond to knowledges and 
skills specified in descriptions of jobs for which the tests are used. 
These studies serve to verify some of our assumptions concerning content 
validity, to uncover some areas in which the assumptions may not hold, 
and to quantify relationships betweeu tests and criteria. 

There are some built-in difficulties in attempting to correlate profi- 
ciency tests with job proficiency. These tests have a strong tendency 
to eliminate applicants who are incapable of performing at a satis- 
factory level of competence. For example, an applicant vho can type 
well enough to be hired as a typist can probably type well enough to be 
considered at least marginally satisfactory on a job requiring typing. 
Thus a dichotomy of satisfactory vs unsatisfactory performance such as 
that used as a criterion in Study III in this report fails to show any 
validity at all for the typing test used. To a lesser extent, the 
decreased variability in skills of successful job applicants in Studies I 
and II must have reduced the magnitude of some of the test-criterion 
relationships that were found. 

A second major area of difficulty in clerical proficiency test validation 
concerns the heterogeneity of job duties within presumably equivalent 
jobs. A wide variety of jobs include typing and dictation duties. In 
a study performed in 1951, Nelson took his job definitions from the 
Dictionary of Occupational Titles and restricted his sample to women 
under 30 with less than three years experience on the job. However, he 
found that there was an extremely wide range in time spent on duties. 

For example, estimates on dictation time ranged from 5 percent to 
90 percent (Nelson, 1951). It is not likely that the same proficiency 
of performance is required for such widely different positions. 

Nelson also pointed out that adequate job analysis requires closer exami- 
nation of the many related elements of work other than dictation and 
transcription speed and errors and that duties listed most frequently 
are not necessarily the critical ones for job success. Nelson was not 
able to investigate the reliability of criterion measures because 
employers refused to re-rate employees. 

Irol Whitmore Balsley concluded that the speed of producing mailable 
copy on the job depended on a number of factors including the speed of 
correcting typing errors (Balsley, 1956). Supervisors' ratings reflect 
an individual's total performance and, in view of Balsley 's findings, 
are not likely to relate exclusively to test measured performance. 
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In two of the three studies reported here, there is demonstrable 
criterion-related validity of the USES Dictation, Spelling and Typing 
Tests for predicting job performance. The third study served to show 
that a simpli satisfactory vs unsatisfactory criterion was not suitable 
for this type of research because of the preselection that necessarily 
takes place in clerical jobs. 
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STUDY I 

VALIDATION OF THE NEW USES SPELLING 
AND DICTATION TESTS (FORM A) 



Summary 

This report describes a concurrent validation study of USES Spelling and 
Dictation Test performance against supervisory ratings of job performance 
of employed stenographers and secretaries. The relationships between 
Dictation Test performance and criterion scores were moderately low and 
positive (.16 to .32). Relationships between Spelling Test performance 
and the criterion scores were low to negligible (.05 to .21). The 
interrelationship of the Spelling and Dictation Test scores was .47. 

Purpose 

The purpose of this study is to determine the relationship of scores on 
the USES Spelling Test and the new USES Dictation Test (Form A) and 
j supervisory ratings of job performance for employed workers. 



Employed stenographers and secretaries — 124 of whom were included in the 
normative study (U.S. Department of Labor, October 1968) and 60 additional 
subjects— were tested with Form A of the USES Spelling and Dictation 
Tests according to standard directions in the Manual for USES Clerical 
Skills Tests (U.S. Department of Labor, March 1968). The Dictation Test 
was administered at 80 wpm on a "live" basis to groups that ranged in 
size from 4 to 15 persons. Raw scores and decile scores were recorded 
for both tests. Data were collected for sex, age in years, education 
in years, and job tenure in the following categories: (1) under 6 months, 

(2) 6 months to 1 year, (3) over 1 year to 3 years, (4) over 3 years to 
10 years, and (5) over 10 years. 

Three criterion measures were obtained for each subject, as follows: 

Part I (ratings on taking and transcribing dictation). Part II (ratings 
on over-all job proficiency), and Total Criterion (Parts I and II com- 
bined). The experimental descriptive rating scale is shown in Appendix A. 

Results and Discussion 

Means and standard deviations are shown on Table 1-1. The fact that the 
mean Dictation Test raw score is less than one standard deviation below 
the maximum score of 125 indicates th*t the Dictation Test at 80 wpm is 
easy for this employed worker sample. The decile mean score of 5.5 is 
the same as the mean for the nation-wide sample upon whom the decile 
scoring system is based (U.S. Department of Labor, October 1968). 
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Table 1-1 

Means and Standard Deviations (N ■ 184) 







Means 


Standard 

Deviations 




Age (years) 


28.1 


11.8 


Sample Characteristics 


Education (years) 


12.3 


1.3 




Job Tenure* 


3.1 


1.1 




Dictation Raw Score 


113.4 


14.3 


Test Score 


Dictation Decile Score 


5.5 


2.8 




Spelling Decile Score 


5.3 


2.7 




Criterion Part I Score 


11.0 


rl.8 


Criterion Scores 


Criterion Part II Score 


19.5 


3.4 




Total Criterion Score 


30.6 


4.9 



* Job tenure is expressed on a 1 to 5 basis , as follows: 1 -- under 

6 months, 2 —6 months to 1 year, 3 --over 1 year to 3 years, 

4 — over 3 years to 10 years, 5 — over 10 years. 
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Table 1-2 shows correlations between biographical data and scores for 
criteria and tests. The relationship between Job Tenure and all three 
criterion scores is statistically significant. Job Tenure does not 
correlate significantly with Dictation Test decile scores. There are 
significant correlations of age and tenure with Spelling Test decile 
scores . 



Table 1-2 

Correlations of Age, Education, and Job Tenure with 
Criterion and Decile Test Scores (N » 184) 





Criterion^ 
Part I 


Criterion 
Part II 


Total 

Criterion 


Spelling 

Decile 


Dictation 
Decile ' 


Age 


.14 


.10 


• 12 


.34** 


.02 


Education 


.10 


.16 


.15 


.12 


-.06 


Job Tenure 


.31** 


.30** 


.33** 


.20** 


.06 



** Correlation significantly different from zero at the .01 level. 



Validity coefficients are shown on Table 1-3. Both the Dictation Test 
raw score and the Dictation Test decile score have a significant corre- 
lation of .32 with Part I (ratings on taking and transcribing dictation). 
Both Dictation Test scores have smaller but equally significant correla- 
tions with the Total Criterion. The two Dictation Test scores have less 
significant correlations with Part II (ratings on overall job proficiency) . 





Table 1-3 



Correlation between Test Scores and 
Criterion Scores (N - 184) 



Test Score 


Criterion 
Part I 


Criterion 
Part II 


Total 

Criterion 


Spelling Decile Score 


.21** 


.05 


.11 


Dictation Raw Score 


.32** 


.14 


.22** 


Dictation Decile Score 


.32** 


.16* 


.24** 



* Correlation significantly different from zero at the .05 level. 
** Correlation significantly different from zero at the .01 level. 



The results Indicate that the transformation of Dictation Test raw 
scores, which are very skewed, to deciles has no effect on the validity 
of the test. Previous research with the USES Typing Test had indicated 
that converting raw scores to deciles had little or no effect on reli- 
ability (U.S. Department of Labor, February 1968) . The Spelling Test 
decile score correlates .21 (significant at the .01 level) with Part I. 
Correlations between Spelling Test decile scores and the other criterion 
scores are not statistically significant. The intercorrelation of 
Spelling Test decile score and Dictation Test decile score is .47. 

Conclusions - 

1. There is a moderately low positive relationship between Form A of 
the new USES Dictation Test scores of employed workers and super- 
visors' ratings. 

2. The relationship between the Dictation Test and the dictation- 
related criterion is greater than the relationships between the 
Dictation Test and other criteria. 

3* There is some relationship between Spelling Test scores and super- 
visors' ratings of ability to take and transcribe dictation. 
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4. There were low positive relationships between tenure and the three 
criterion measures used in this study. The relationships between 
tenure and the criteria appear to be relatively independent of the 
relationships between Dictation Test scores and the criteria. 

Recommendations for Future Research 

1. Validation of other forms of the Spelling and Dictation Tests is 
recommended. 

2. Validation research with an applicant population using a longitudinal 
design is also recommended. 
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STUDY II 

FOLLOW-UP STUDY ON THE USES TYPING AND 
SPELLING PROFICIENCY TESTS 



3uwnarv 

This report describes a longitudinal study of applicants' performance on 
USES Typing and Spelling Tests and subsequent supervisory rating measures 
and retest performance scores obtained on the job. Results show that 
typing speed scores and spelling scores are stable and correlated measure 
ments and that typing speed scores predict performance on jobs requiring 
typing. Applicants' typing error scores did not predict typing error 
scores on the job or the supervisory rating measurements. 

Purpose 

The purpose of this study is to determine predictive validity and reli- 
ability of the USES Typing and Spelling Tests. 

Procedure 

Local office applicants were tested with the USES Typing and Spelling 
Tests prior to placement, placed in jobs requiring at least one hour of 
typing per day, and followed up on the job 90 to 120 days after initial 
testing. Subjects who had been on the job at least 60 days were retested 
and job performance ratings were obtained from supervisors. 

Any form of the typing test was used and administered according to 
"Directions for Administering and Scoring the New USES Typing Test" 

(U.S. Department of Labor, March 1968). Subjects were retested on the 
employer's premises provided that the employee was using the same kind 
of typewriter (that is, manual or electric) that he used as an applicant 
examinee. If this condition could not be met, the subject was eliminated 
from the sample. Subjects were tested initially with Form T62-M of the 
Spelling Test and retested with Form T63-M. Both tests were administered 
according to standard directions (U.S. Department of Labor, 1953). 

At the time of retesting, a job performance rating was obtained from the 
Immediate supervisor using Form SP-21 (U.S. Department of Labor, 1967). 

The information recorded for each subject is listed in Appendix B. 

A copy of the experimental design is attached in Appendix C. 
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Results and Discussion 

Table II-l shows means and standard deviations for 19 variables. The 
113 subjects had a mean age of 26 years and approximately three and one- 
half years experience in jobs requiring typing. Subjects in the normative 
sample had a mean age of 33 years. The normative sample was representa- 
tive of all employed Secretaries, .Stenographers, and Typists in the U.S. 
(as reported in the 1960 Census of Population) on geographic distribution, 
age, education, sex, and industry in which employed (U.S. Department of 
Labor, March 1968) . 



Table II-l 

Means and Standard Deviations for 
Nineteen Variables (N • 113) 



Variable 

Number 


Variable 


M 


SD 


1 


Age (years) 


25.9 


8.8 


2 


Education (years) 


12.9 


1.4 


3 


Semester or quarters of typing training 
completed 


4.4 


2.3 


4 


Time between initial testing and last 
typing training (months) 


64.5 


88.0 


5 


Experience in jobs requiring typing 
(months) 


43.9 


61.1 


6 


Time between initial testing and last 
typing job (months) 


16.1 


49.7 


7 


Variable No. 4 or No. 6, whichever is 
lower 


14.4 


38.6 


8 


Initial typing test score -- wpm 


53.2 


12.3 


9 


Initial typing test score — errors 


7.0 


3.9 


10 


Initial spelling test score (T62-M) 


66.0 


9.9 


11 


Retest typing test score — wpm 


59.1 


12.9 


12 


Retest typing test score -- errors 


9.9 


6.2 


13 


Retest spelling test scores (T63-M) 


72.0 


8.5 


14 


Typing speed change ) Retest score 


106.0 


6.6 


15 


Typing accuracy change ) minus initial 


102.9 


6.8 


16 


Spelling change ) score plus a 

) constant of 100 


106.0 


6.4 


17 


Hours per day spent typing on job on 
which followed up 


3.8 


1.8 


18 


Experience on job on which followed 
up (weeks) 


11.7 


2.9 


19 


Supervisory rating score (Form SP-21) 


25.1 


4.7 



15 -r 
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Mean retest scores were higher than mean Initial test scores. No direct 
comparison can be made with the normative sample for which data Is listed 
separately for manual and electric typewriter subjects. However, the 
mean Initial wpm score of 53.2 Is lower than both means computed for the 
normative sample. 

Supervisory rating scores were normally distributed around the mean of 
25.1. Scores ranged from 11 to 35 with a standard deviation of 4.7. 

Age, experience In jobs requiring typing, and three variables on time 
since training or experience are concentrated at the lower ends of the 
scales. Education Is highly concentrated at 12 years. 

Table II-2 shows the number and percent of subjects listed by D.O.T. 
code number for this study and for Study III--Relatlonshlp Between 
Performance on USES Typing Test and Acceptability of Typing Performance 
to Employers, which Is described later In this report. The D.O.T. codes 
represented In both studies and the percentage of cases In each study 
having each D.O.T. code number are very similar. 



Table II-2 

Number and Percent of Subjects Listed by 
D.O.T. Code Number for Study II and Study III 



D.O.T. 
Code Number 


Study II 


Study IIT 


Number 


Percent 


Number 


Percent 


201.368 


20 


18 


74 


16 


202.388 


15 


13 


70 


15 


203.588 


2 


2 


34 


7 


204.288 


1 


1 






206.388 


1 


1 






208.588 . 


6 


5 


11 


2 


.209.388 


46 


41 


185 


39 


209.588 


2 


2 


29 


6 


210.388 


1 


1 


1 


(+) 


219.388 


15 


13 


29 


6 ! 


219.488 


2 


2 


1 


(+) 


237.368 


1 


1 


3 


(+) 


249.368 


1 


1 


2 


(+) 


Other D.O.T. Codes* 


- ■ 


- 


30 


9 


Totals 


113 


101** 


469 


100 



(+) A fraction of one percent. 

* Additional six^diglt D.O.T. codes plus 19 Incomplete D.O.T. codes are 
listed In Study III. 

** Total percent equals 101 due to rounding. 
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Intercorrelation data for the 19 variables In this study are shown on 
Table II-3 . No significant correlations were found for education or 
for hours per day spent typing. 



Variable List 



1. Age (years). 

2. Education (years). 

3. Semesters or quarters of typing training completed. 

4. Time between Initial testing and last typing training (months) . 

5. Experience In jobs requiring typing (months). 

6. Time between Initial testing and last typing job. 

7. Information listed under Variable No. 4 or Variable No. 6, 
whichever Is lower. 

8. Initial typing test score in number of words per minute. 

9. Initial typing test score— number of errors. 

10. Initial spelling test score (Form T62-M) . 

11. Retest typing test score in words per minute. 

12. Retest typing test score— errors . 

13. Retest spelling test score CT63-M). 

14. Typing speed change: retest words per minute score minus 

Initial typing test words per minute score plus a constant 
of 100. 

15. Typing accuracy change: retest error score minus initial error 

score plus a constant of 100. 

16. Spelling change: retest spelling test score minus initial 

spelling test score plus a constant of 100. 

17. Hours per day spent typing on job. 

18. Experience on job on which followed up (weeks). : 

19. Supervisory rating score (Form SP-21). 



The time variables (age, experience in jobs requiring typing, time since 
last typing training, time since last typing job, and time since typing 
training or typing job) are intercorrelated. Also, semesters or quarters 
of typing training is correlated with experience in jobs requiring typing 
In general, the correlations between demographic variables and both test 
score variables and supervisory ratings are low. Experience in jobs 
requiring typing, however, correlates significantly with 10 of the 19 
variables, and significant correlations with typing speed and spelling 
scores were. found ranging from .33 to .46. The significant relationship 
of experience and test score variables is compatible with the observation 
that the employed worker normative sample (which is seven years older on 
the everage and presumably more experienced in jobs requiring typing) has 
higher typing speed mean scores than the applicant sample. 





ERIC 



Test-retest correlations are .87 and .77 for typing wpra scores and for 
spelling test scores, respectively. Significant correlations of .39 
and .41 were computed between typing wpm scores and spelling test scores. 

No significant correlation wa6 found between initial and retest typing 
error scores. Retest typing error score has a -.18 correlation signifi- 
cant at the .05 level with experience in typing jobs. 

Typing accuracy change correlated with initial and retest typing error 
scores. Spelling change correlated, with initial spelling test scores. 

Like hours-per-day spent typing, experience on the job on which followed 
up was used as a control variable. Subjects with less than eight weeks 
on the job were not included in the sample. However, five variables 
correlate significantly with experience on the job on which followed up. 
This variable correlates .33 (significant at the .01 level) with initial 
typing speed score. 

Supervisory rating score has low correlations with age, time since typing 
training, experience in jobs requiring typing, retest spelling scores, 
and experience on jobs on which followed up. Supervisory rating score 
correlates .44 with initial typing wpm score and .36 with retest typing 
wpm score. Supervisory ratings did not correlate significantly with 
typing error scores. 

Conclusrtons 



1. The correlations obtained between typing wpm scores and supervisory 
rating scores are evidence of validity of typing test speed scores 
as predictive of performance on jobs requiring typing skill. 

2. There is good stability of measurement between initial test and retest 
performance for typing wpm scores and spelling test scores. 

3. There is no apparent stability of measurement between initial test 
and retest performance on typing error score. 

4. There is a moderate but highly significant correlation between typing 
wpm scores and spelling test scores. 

5. Experience in jobs requiring typing is correlated with ten other 
variables including typing speed scores, spelling scores, and super- 
visory ratings. 



Reconmendat Ions 



1. The finding that typing error score does not correlate with test 
performance on the Job or with supervisory ratings should receive 
further investigation. If this finding is supported by additional 
independent study, typing error score should be eliminated or altered 
in reporting typing test performance results. 

2. In the future, data should be collected separately for applicants 
who use manual and electric typewriter. 

3. In view of the finding that semesters or quarters of typing training 
completed, time since last typing training, time since last typing 
job, and time since last training or job do not add anything to the 
standard background variables (age, education, and experience), they 
should not be included in future studies. 

4. The Supervisory Rating (Form SP-21) appears to be an adequate instru- 
ment for rating typing performance on the job and should be considered 
for use in future studies. 
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STOTT HI 

RBLATX0S8HXP BKTVSEV FSBPORNAVCS OV USES TZPXM TEST 
AID ACCKPEABZLITr OF TI PI Ett FESFOBMAICE TO BIFIOZnS 



This report describee e longitudinal study of applicant* ' typing test per- 
fornaaoe end subsequent supervisors ' evaluations of typing perforasnee on 
the Job. So significant validity coefficients cere found. Ho w ever, findings 
do show tbat typing skill Is essential for a vide range of Jobs that do not 
necessarily require the sane level of proficiency la typing. Also, findings 
Indicate the need for preliminary study of criterion measures to Insure ade- 
quate variance for statistical traatnant. 

Purpose 

The purpose of this study Is to detemlas predictive validity of the OSES 
Typing Test. 




The OSES Typing Tact vas adntnlatered to applicants for positions requiring 
typing aklll. State agencies attenpted to follow up each applicant 90 to 120 
days after testing and to obtain typing performance evaluations fnan super* 
visors for those subjects who bad been employed at least eight weeks on a Job 
requiring a minimus average number of two hours typing psr day on the sans 
kind of typ ewrite r (nanual or electric) as the one used for testing. Any 
fora of the USES Typing Test was adninlstered according to "Directions for 
Adninlstorlng and Scoring" In the Manual (U. 3. Department of Libor, March 1968). 

Data were collected for 15 varlablos listed In Appendix D. 

The experimental plan la la Appendix B, and the Evaluation Sheet used for 
recording criterion data la In Appendix F. Supervisors were asked to rate 
subjects aa "Fully satisfactory" or "uasatlsfaetovy or only barely satisfactory" 
on three asperate variables: spaed of typing, accuracy of typing# and overall 
typing perfomance. 

Results and Discussion 

State agencies tested l,ltd applicants. Forty-one percent of the tested 
applicants vara subsequently evaluated by supervisors. Table HI-1 shows for 
each State and for the combined sample the umber of applicants tested, the 
number eliminated from evaluation (listed in seven descriptive categories), 
and the amber of subjects in the final sample for Whoa supervisor's evalua- 
tions were obtained. The mbit frequent reason for eliminating subjects from 
the final sample was that the type of typewriter used on the Job was different 
from the one on which tested. 
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Table III-l 

Number of Subjects Tested, Eliminated from Subsequent 
Evaluation Defined in Seven Categories, and Evaluated by 
An Employer Listed by State and for the Combined Sample 



State 


Number of Subjects 


Tested 


Eliminated 


Evaluated 


1 _ 


2 








6 


zn 


Lack of Cooperation 
by Employer 


Subject Never Employed 


Subject No Longer Employed 
Supervisor not able to Rate 


Subject Had Not Completed 
Training Period 


Typing Not Required on 
Present Job 


Type of Typewriter Used on 
Job (Manual or Electric)# 


Other Reasons for Elimina- 
tion 


California 


128 


0 


9 


63 


6 


11 


7 


1 


31 


Georgia 


80 


0 


0 


12 


1 


15 


12 


1* 


36 


Illinois 


58 


0 


1 


1* 


0 


3 


10 


0 


1*0 


Louisiana 


130 


0 


37 


0 


20 


8 


20 


1* 


1*1 


Minnesota 


U5 


0 


1* 


1* 


0 


9 


31* 


22 


1*2 


North Carolina 


no 


2 


6 


6 


2 


8 


1*7 


0 


39 


Olio 


13k 


9 


3 


18 


1* 


12 


51 


5 


32 


Oklahoma 


85 


12 


0 


7 


12 


6 


8 


0 


1*0 


Rhode Island 


1*3 


0 


2 


2 


3 


5 


0 


0 


31 


Texas 


85 


0 


1 


12 


0 


8 


5 


16 


1*3 


Virginia 


113 


2 


3 


6 


11 


13 


21 


10 


1*7 


Washington 


67 


0 


0 


0 


0 


5 


13 


2 


1*7 


Totals 


111*8 


25 


66 


131* 


59 


103 


228 


61* 


1*69 


Percents-*# 

it - - — - 


101 


2 


6 


12 


5 


9 


20 


6 


1*1 



♦Different from -type on which tested 
■a# The figure 101 is due to rounding 
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Table XXI-2 sbova Mans sad standard drrlatlona for age, education, Job 
experience, typing apoad score, typlag accuracy score, and average mober of 
hours per day spent typlag for subjects who were evaluated by supervisors aad 
available comparable data for the national normative a ample vhlch vaa repre- 
sentative of all employed secretaries, stenographers, sad typists in the 
United States (as reported la the i960 Census of Population) on geographical 
distribution, age, education, sex, and Industry in vhlch employed. Data for 
both groups Is listed separately for subjects tested on manual typewriters aad 
subjects tested on electric typewriters. 

fable m-2 

Naans aad Standard Deviations Listed Separately for Manual aad 
Electric typewriter Subjects on Six Variables for Study IZZ and 
for Available Data for the Normative Sample 



Variable 


Study m 


Noraatlve Sample* 


Manual (1-197) 


Electric (E-272) 


Manual (V-88l) 


Electric (1-881) 


M 


SD 


N 


SD 


M 


SD 


M 


SD 


Age (years) 


24.8 


8.6 


25.8 


9.0 


33.8 




-33.4 _ 




Education (years) 


12*5 


1.0 


12.7 


1.2 


12.4 




12.5_ 




Job Experience faonthsl 


28.2 


41.7 


37.3 


47.3 










typlag Speed Scorefo.) 


50.5 


10.4 


58.9 


10.7 


99.0 




70.0 




typing Accuracy Score 


7.4 


5.4 


6.8 


5.0 


8.1 


6.2 


9.0 _ 


- T*1 


Hours per day + 


4.0 


1.6 


■JtiS 


1.6 











«U. 8. Department of Labor, March 1968, pp. 17*18. the sample was representative of all 
anployed Secretaries. Stenographers, aad typists In the U. S. (as reported In the i960 
Census of Population) on five variables t geographical distribution, age, education, sex, 
aad Industry In which employed. 

♦ U. S. Department of Labor, November 1964. Subjects in the noraatlve sample typed at 
least one hour per day. 
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The appllcast subjects are considerably younger on the average than employed 
workers. Both applicant groups have lower typing speeds than the normative 
sample groups# 

Typing applicants were employed In eight major Industries# Slnllar data on 
Industry In which employed for subjects In the follow-up study and employed 
workers In the nomat Ire sanple are shown on Table III -3* 



Table III-3 

Number and Percent of Ctses by Industry for Study III 
and the Normative Sample 



Industry* 


Study lit 1 

(N-U69) 


Normative Sample** 
(N-1762) 


Number 


Percent 


Number 


Percent 


Agriculture, forestry and fisheries 






1 




Mining 


U 


1 


11 


1 


Construction 




1 


3 




Manufacturing 


99 


21 


Shh 


31 


Transportation, communication 










and public utilities 


19 


k 


95 


5 


Wholesale and retail trade 


60 


13 


21b 


12 


Finance, insurance and real estate 


89 


19 


328 


19 


Business and repair services 


19 


U 


23 


1 . 


Personal services 






12 


1 


Entertainment and recreation 


1 




2 




Professional and related services 


81 


17 


216 


12 


Public administration 


92 


20 


313 


18 



♦Data were collected by industry according to the Standard Industrial 
Classification Manual # (Bureau of the Budget, 19 67) 

**U# S# Department of Labor, November 196U# 
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Table XIX-U shows number of eases by D.O.T. oode numbers and average number of 
hours per day spent typing for manual typewriter subjects and electric type- 
writer subjects. Actual D.O.T. six digit code numbers and Job titles are 
listed In Appendix 0. In general, a six digit D.O.T. code number does not 
refer to a specific Job but to many different Jobs. lours per day spent 
typing varies widely within each occupation. 
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Table III-l* 

Number of Cases Listed By D.O.T. Code Number and Hour’s Per Day Spent Typing 
for Manual and Electric Typewriter Subjects 



! 1 

D.O.T. 

Code* 


Manual Group (N«*197) 


[ Electric Group (N=272) 


7! 

H Eh 


Hours Per Day Spent 


Typing 


. ^ Eh 


! Hours Per Day S 


pent Typing 


Total 

DOT 


3 o’ 

£ Q 


2 


3 


h 


5 


6 


7 


8 


■P o 
O Q 
Eh 


12 

i 


3 


k 


! 5 


6 


7 


8 


1 




1 












l 


1 ' 
i 
















1 


2 




















1 


1 










1 


1 


3** 


3 


1* 


3 


5 


1 


1 




17 


1* 


k 


18 


11 


18 


2 




& 


71* 


U** 


1* 


3 


3 


3 








13 


6 


12 


15 


j 10 


12 




2 


57 


70 


5 






1 










1 


















1 


6 


3 


3 


1 


2 


2 


1 


1 


13 


1* 




l 


3 


U 


3 


6 


21 


31* 


7 


2 




1 


l 


2 






6 




2 




2 


1 






5 


11 


8 












1 




1 


















1 


9 


15 


16 


26 


15 


12 


1* 


2 


90 


13 


20 


22 


15 


18 


5 


2 


95 


185 


10 


3 


3 


1 


2 


3 


1 




13 


1* 


2 


2 




6 


2 




16 


29 


11 


















1 














1 


1 


12 








1 








1 


















1 


13 


1 














1 


















1 


ii* 










1 






1 


















1 


15** 


5 


8 


5 


2 


1 


1 


1 


22 




1 


2 




3 




1 


7 


29 


16 












I 








1 












1 


1 


17 


















1 














1 


1 


18 


l 














1 


















1 


19 


l 






1 








2 








l 








1 


3 


20 


l 


1 












2 


















2 


21 




1 












1 


















1 


22 




i 












1 


















1 


23 


2 


2 


2 


2 




2 




10 


2 


1 


li 




2 






9 


19 


Total 

Hours 


Ui 


1*3 


1*3 


31* 


22 


10 


1* 

. . 


197 | 


35 


10* 


6U 


1*2 


61* 


12 


11 


272 


1*69 



^Numbers refer to D.O.T. codes shown in Appendix G. 



**The distributions for manual and electric typewriter subjects differ signifi- 
cantly on occupation at the .01 level. 
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Chi square vas computed for the distribution of Manual and electric typewriter 
subjects for each D.O.T. code. For three codes, the predonlaant use of oae 
kind of typewriter was significant at the .01 level. Two categories of steno- 
graphers are nore likely to use electric Machines j one category of clerks is 
■ore likely to use nsnual typewriters. 

Typing speed nsans and standard deviations were conputed for subjects la four 
D.O.T. codes. Data on Table HI-5 shows that subjects having D.O.T. codes 3 
and 4 have higher typing nsan speeds than subjects in D.O.T. codes 9 and 15 
and subjects in the total sanple. Host subjects in codes 3 and 4 are stenographers 
and secretaries. Various kinds of clerks are coded 9 and 15. 

Table IXI-5 

Typing Speed 8eore Means and Standard Deviations Listed Separately 
for Manual and Electric Type wri ter Subjects for Four Sanples Defined 
by D.O.T. Code Reference Amber and for the Total Sanple in Study HI 





Manual subjects 


Electric subjects 


Sanple * 


V 


M 


SD 


V 


M 


SD 


D.O.T. Code 3 


17 


5 6.4 


12.4 


57 


61.7 


11.4 


SeOfT# 0o4® ^ 


13 


56.2 


6.2 


57 


61.4 


9*5 


D.O.T. Code 9 


90 


50.0 


9.9 


95 


53.1 


10.0 


D.O.T. Code 15 


28 


*19.3 


12.5 


7 


51.6 


12.9 


Total (frfenSy m) 


197 


50.5 


10.4 


272 


58.9 


10.7 



*Cbnplete D.O.T. code numbers and Job titles are listed in Appendix D. 



Correlations of ordered variables are shown on Tables HI-6, IH-7, and HI-8 
for the total sanple, nanual subjects, and electric subjects, respectively. 
The correlation between type of typewriter used and typing speed is .36 
(significant at the .01 level). 
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■S 

i 

i 

t 

i 

j 

! ■ 
f 
i 

mu m-6 



Product Mo— nt Correlation Coefficients for the Total Staple oa 
Eleven Tailables (1469) 





1 


$ 


1 

I 


ii 


1 

£ 


* 

2 

S 


i* 

11 


1 

ii 

If 


3 

m 8 

H 

! 

Us 


i 

•3 

if 

il 

US 


In 


fOT 


«* 

.68 


** 

.13 


-.04 


.06 


.04 


-.02 


• 

.10 


-.02 


.02 


Bducet ion 




.01 


«* 

r 20 


- -.03 


.or 


-t91 


.08 


-.05 


-.04 


-.05 ! 


Job Experience 






** 

•2b 


.04 


• 

.10 


.05 


-.01 


.01 


-.OJ 


—01 1 


Erplat Sneed Score 








-.03 


.36 


.09 


-.05 


* 

-.11 


-.09 


-.08 ! 


Imping Accuracy 
Score 










-.05 


-.02 


-.03 


-.02 


.00 


! 

-.03 


Erne of Typewriter 












«* 

.16 


.00 


-.02 


-.03 


11 i 

-.02 1 


lours Per Say 














.05 


-.01 


.00 


-.02 


Experience ea 
Present Job 
















-.02 


.01 


.03 


Supervisors lvalue* 
tloa Speed 


















*• 

.53 


*» 

.61 


Supervisor's lvalue* 
tloa Accuracy 




3 
















- ! 

.79 



* Correlation significant at the .0$ level 
••Correlation significant at the .01 level 
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Table III-7 

Product-Moment Correlation Coefficients for Manual Typewriter 
Subjects for Ten Variables 
(N-197) 











t 

A 




c 

Hi 


TJ 










0) 
t \ 


£ 

O 

o 

CO 


V 

o 

CO 

rt 

U 

*3 




w 

1 

#4 


© 

© 

a 

CO 










s 


Tf 

<D 


a 


o 


ra G 


•• 

© c 


•• 

CO G 




Education 


I 

1 


s, 

CO 

to 

.5 

B 


p— 

o 

% 

to 

g 

•d 

J* 


u 

a! 

2 

o 


0) 

G 

© 

© o 

r 


1 Supervisor* 
Evaluatic 

i 

1 


Supervisor 1 

Evaluatic 

Accuracy 


Supervisor 1 








* 




* 










Age 


.13 


.73 


.18 


-.07 


.16 


.03 


.03 


-.06 


.03 


Education 




.09 


.13 


-.01 


.01 


-.07 


-.10 


-.01 


-.03 








#«• 




• 










Job Experience 






.25 


.03 


.17 


-.01 


.05 


.00 


.11 












* 










Typing Speed Score 








.09 


.15 


.03 

j 


-.06 

1 


-.08 


-.06 


Typing Accuracy Score 










.02 


-.12 


I 

• 

o 

vn 


-.01 


-.06 


Hours Per Day 












.11 


.06 


.06 


.00 


Experience on Present 
Job 














; .02 


.01 


.00 


Supervisor's Evaluation: 
Speed 










1 

1 




j 


•U9 


•a?* 

.61 


Supervisor’s Evaluation: 
Accuracy 














i 


I 

-J 


.68 



^Correlation significant at the .05 level 
^Correlation significant at the .01 level 
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Table III-8 



Product-Moment Correlation Coefficients for Electric Typewriter 

Subjects for Ten Variables 
(N-272) 





Education 

l— -e 


a 

§ 

a> 

I 


Typing Speed Score 


Typing Accuracy 
Score 


Hours Per Day 

- 


Experience on 
Present Job 


Supervisor's 
Evaluation: Speed 


Supervisor's 

Evaluation: 

Accuracy 


[Supervisor's 

i Evaluation: 

1 ftreyaTI 


Age 


.03 


*# 

.65 


.07 


1 

• 

o 

H 


-.05 


-.05 


.15 


.01 


.01 


Education 




-.oU 


.23 


-.05 


-.05 


-.08 


-.01 


-.06 


-.06 


Job Experience 






.20 


•06 


-.05 


-.01 


-.01 


-.08 


-.08 


Typing Speed Score 








-.10 


-.05 


-.11 


-.15 


-.10 


-.09 


Typing Accuracy Score 










-.05 


.01 


-.01 


.00 


-.01 


Hours Per Day 












.02 


-.05 


-.03 


-.02 


Experience on Present 
Job 














-.05 


.01 


.05 


Supervisor's Evaluation: 
Speed 
















** 

.57 


** 
• 60 


Supervisor's Evaluation: 
Accuracy 


















** 

•89| 



^Correlation significant at the .01 level 
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The three criterion scores here moderately high lntercorrelstloms. For each of 
three samples, the highest criterion score intercorrelat ion is between super- 
visor's accuracy eraluatlon and supervisor's orerall evaluation. The statistical 
findings support the general experience of State agencies that employers place 
highest value on typing accuracy. 

Only 6l out of 469 subjects or 13)1 of the total sample were rated unsatisfac- 
tory on one or nore criterion variables. The distributions for criterion 
variables are shovn, as follovt: 



Supervisor's Evaluation 



Bating 


Manual Groups 
(■-197) 


Electric Group 
(E-272) 


Total 

( 1 * 4 < 


Sample 

S 9 ) 




Accuracy 


overall 




Accuracy 


orerau 




Accuracy 


orerau 


Satisfactory 


187 


179 


181 


261 


251 


253 


448 


430 


434 


^satisfactory 


id 


10 


16 


11 


£1 


19 


21 


3 $ 


37 



Little or no predictive validity was found between typing scores and supervisor's 
evaluations. Absence of validity any be attributed to a number of factors, as 
follows: 1 

I 

1. Subjects had been selected partly on the basis of test performance. j 

2. Unsatisfactory typists any have been discharged or moved to positions J 

not requiring typing prior to evaluation. 

1 

3. The wide variety of Jobs represented In the sample do not necessarily 

require uniform proficiency in typing. j 

4. There is very little variance In the criterion measures. j 

j 

Standard industrial Classification, D.O. T. code number, and employer! s speci- 
fications on speed and accuracy were also treated as predictors by applying 
chi square to the difference in frequencies occurring for satisfactory and 
unsatisfactory subjects. A subject who was rated unsatisfactory on any cri- 
terion variable was categorised as "unsatisfactory". 





i 
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Table HI-9 above frequencies and percents for unsatisfactory and satisfactory 
subjects in eight major industries. So significant differences vere found 
among Industries with respect to the proportion of individuals evaluated as 
unsatisfactory. 

Table XXI-9 

■tuber and Percent of Subjects Listed by industry and Supervisor's 
■valuation for the Total Sample (9*k69) 





unsatisfactory" 

Subjects 

(■*1) 


Satisfactory*" 

Subjects 

(Bb4o8) 


Total 

Sanple 

(■-489) 


j 


Industry 


ffo&ber 


Percent 


ftadber 


Percent 


feni her 


Percent 


Mining 






b 


1 


4 


i 1 


Construction 




5 


2 


1 


5 


i 

1 1 


Manufacturing 


n 


18 


88 


21 


99 


. i 


Transportation, 
coosumloatlons and 
public utilities 


k 


7 


15 


if 


19 


4 


Iholesale and retail 
trade 


8 


13 


52 


13 


60 


13 i 


Finance, real estate 


Ik 


23 


75 


18 


_ 89 


i 

i 

19 i 


kKrieo* 


Ik 


23 


87 


21 


101 


22 


Public administration 


7 


11 


85 


21 


2* 


i 

20 



"Unsatisfactory subjects are those rated unsatisfactory on one or more criterion 
variables, namely: Supervisor's Svaluat Ion-Speed , Supervisor's Evaluation- 

Accuracy, and Supervisor's Evaluation-Overall. 

""Satisfactory subjects are those rated satisfactory on all three criterion 
variables. 



Table XXX-10 shows frequencies and percents for unsatisfactory and satisfactory 
subjects having 23 different D.O.T. code numbers. Systematic differences vere 
found. One group of court reporters and stenographers are more likely to be 
rated satisfactory. Subjects having an "X" in the D.O.T. code (entry level 
employees) are more apt to be rated unsatisfactory by supervisors. 
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Table III-10 

Number and Percent of Subjects Listed by D.O.T. Code Number 
and Supervisor's Evaluation for the Total Sample 

(N-l*69) 



Reference (lumber 
for D*0*T* Code 
Shown in 
Appendix B 


Unsatis 

Subject 


(factory 
« (N*6l) 


Satisfactory 
Subjects (N»l*08) 


Total 

(N- 


Sample 

1*69) 


Number 


Percent 


Number 


Percent 


Number 


Percent 


1 






1 


•2 


1 


.2 


2 






1 


.2 


1 


•2 


- 3 


11 


18.0 


63 


15.0 


7k 


16.0 


*»ll 


2 


3.0 


68 


17.0 


70 


15.0 


5 






1 


•2 


1 


•2 


6 


6 


10.0 


28 


7.0 


3U 


7.0 


7 






11 


3.0 


11 


2.0 | 


8 






1 


.2 


1 


.2 


9 


19 


31.0 


166 


1*0.0 


185 


39.0 


10 


5 


8.0 


2l* 


6.0 


29 


6.0 


11 






1 


.2 


1 


.2 


12 






1 


•2 


1 


•2 


13 


1 


2.0 






1 


.2 


1 k 






1 


.2 


1 


i .2 


15 


■ 1' | 


2.0 


28 


7.0 


29 


6.0 


16 






1 


•2 


1 


.2 


17 






1 


•2 


1 


.2 


18 






1 


•2 


■ 1 i 


.2 


19 






3 


.7 


3 


.6 


20 






2 


.5 


2 : 


• U 


21 






1 


.2 


i 


.2 


22 






1 


•2 


: • i 1 


.2 


**23 


16 


26.0 


! 3 

i 


• 7 


i 19 


! u.o 



**The distribution of unsatisfactory and satisfactory ratings on 
occupation is significantly different from zero at the *01 level* 
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fkbln m-11 and XU-12 ahov frequencies rad percents for unsatisfactory rad 
satisfactory subjects listed accord Inc to enploynrAi speed sad accuracy speci- 
fications* feqployers vbo do not specify speed and recuracy requirements are 
■ore likely to employ typists vho are unsatisfactory to then* 

Table m-11 

Vuaiber and Percent of Subjects Listed by Bpployer'ft Speed Specifications 
and Supervisor's evaluation for the Total Sanple (**469) 



f 

i 

! 

I 

l 



t: 






Bsployer Speed 
Specification 


Unsatisfactory 

Subjects 

(*-6D 


Satisfactory 

Subjects 

(** 08 ) 


Total 

Sanple 

(*■469) 


amber 


Fercemt 


■unbar 


Percent 


■unbar 


Percent { 


e>o specification 


26 


46 


118 


29 


146 


31 


8o vps or acre 














70-79 to 






6 


1 


6 


1 


60-69 vpa 


5 


8 


39 


9 


43 


i 

9 


50-59 vps 


10 


l6 


82 


20 


—98 


20 


40-49 vna 


11 


18 


117 


29 


lie 


! 

27 ! 


39 vpn or less 


1 


2 


—5 


1 


6 


1 1 


Intonation not 
available 


6 


10 


*2 


10 


48 


10 



I 



■The distributions unsatisfactory and satisfactory ratines differ significantly 
at the .01 level. 
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Table XIX-12 

Sober and Percent of Subjects Listed by Snployer's Accuracy 
Specification and Supervisor's Evaluation for the Total Sanple (S»*69) 



r 

Employer's Accuracy 


Qhsatlsl 
Subjec 
. («■*!) 


’actory 

its 


Satisfactory 

Subjects 

t*+Q8) 


Total 

Sanple 

(**69) 


Specification 


lumber 


Percent 


■tniber 


Percent 


Rabtr 


Percent ! 


■So specification 


32 


52 


15* 


38 


186 


39 


Ugh accuracy 


15 


25 


129 


_ 32 


1** 


31 


Average accuracy 


6 


13 


70 


17 


78 


17 


Minimal accuracy 






6 


1 


6 


1 


information not 
available 


6 


10 


_*2_ 


12 


55- 


i 

12 



distributions of unsatisfactory and satisfactory ratings differ significantly 
at the .05 level. 



Conclusions 



1. The applicant sanple used la this study and the normative sanple on vhieh 
the typing test noxns are based have different background and test per- 
formance characteristics . 

2. typing skill is essential for a vide range of Jobs vhlch do not necessarily 
require the sane level of proficiency in typing. 

3. Xvaluatlon of subjects as satisfactory and unsatisfactory on speed, accuracy, 
and overall perfomaace did not result in equally proportioned distribution 
of scores. 

Baconmendatlona 



Future validation research should include test data collection on the total 
applicant sample including subjects vho are eliminated from evaluation. 

Future validation studies should include a preliminary investigation and 
classification of Jobs vith reference to typing duties and typical level 
of proficiency required. 

Future validation studies should include preliminary study of criterion 
measures and selection of one that seems likely to provide optimum variance 
in scores for statistical treatment. 



1 . 



2 . 



3 . 
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Appendix A 

DESCRIPTIVE RATING SCALE (Experimental) 

(For U.S.E.S. Dictation and Spelling Tests Normative Study) 



Score 



RATING SCALE FOR 

D.O.T. Title and Code 

Directions: Please read Form SP-20, "Suggestions to Raters" and then fill in the 

items listed below. In making your ratings , only the one most suitable 
answer should be checked for each question. 



Name of worker (print). 



(Last) 



(First) 



Sex: Mal e F emale 
Company Job Title : 



How often do you see this worker in a work situation? 

( ) See her at work all the time. 

( ) See her at work several times a day. 

( ) See her at work several times a week,* 

( ) Seldom see her in work situation.* 



How long have you worked with her? 

( ) Under one month. 

( ) One to two months. 

( ) Three to five months. 

( ) Six months or more. 



Hew long has worker been employed as a secretary or stenograpner by your firm? 

( ) Under 6 months. 

( ) 6 months to one year. 

( ) 1 to 3 years. 

( ) 3 to 10 years. 

( ) Over 10 years. 




RATED BY TITLE 



NAME OF COMPANY 

LOCATION • DATE. 

(City) (State) 

ES 629.1-1 (8/66) N.Y.-DEPT. OF LABOR-D.E. 
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I. Proficiency in Taking and Transcribing Shorthand Dictation. 

A. How fast can she take shorthand dictation? (Worker's ability to make 
efficient use of time and to work at high speed.) 

( ) 1. Capable of very slow speed. Can perform only at an unsatisfactory 
pace. 

( ) 2. Capable of low speed. Can perform at a slow pace. 

( ) 3. Capable of fair speed. Can perform at an acceptable but not fast 

pace . 

( ) A. Capable of high speed. Can perform at fast pace. 

( ) 5. Capable of very high speed. Can perform at an unusually fast pace. 



B. How accurate is transcription of dictation taken. (Worker* s ability to 

avoid making mistakes.) 

( ) 1. Makes very many mistakes in transcribing dictation taken. Work needs 
constant checking. 

( ) 2. Makes frequent mistakes in transcribing dictation taken. Work needs 
more checking than is desirable. 

( ) 3. Makes mistakes occasionally in transcribing dictation taken. Work 
needs only normal checking. 

( ) 4. Makes few mistakes in transcribing dictation taken. Work seldom 
needs checking. 

( ) 5. Rarely makes mistakes in transcribing dictation taken. Work almost 
never needs checking. 



C. How fast is the person in transcribing her dictation? (Ability of person to 
work rapidly.) 

( ) 1. Capable of very low output. Can perform only at an unsatisfactory 
pace* 

( ) 2. Capable of low output. Can perform at a slow pace. 

( ) 3. Capable of fair work output. Can perform at an acceptable but not. 

a fast pace. 

( ) 4. Capable of high work output. Can perform at a fast pace. 

( ) 5. Capable of very high work output. Can perform at an unusually fast 

pace. 
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II. Worker »s Over-all Job Proficiency 

A. How much does she know about her job? (Worker* s understanding of the 
principles, equipment, materials and methods that have to do directly or 
indirectly with her work.) 

( ) 1. Has very limited! knowledge. Does not know enough to do her Job 
adequately. 

( ) 2, Has little knowledge. Knows enough to "get by.'* 

( ) 3. Has moderate amount of knowledge. Knows enough to do fair work. 

( ) 4. Has broad knowledge. Knows enough to do good work. 

( ) 5. Has complete, knowledge. Knows her job thoroughly. 



B. How much aptitude or facility does she have for this kind of work? (Worker * s 
adeptness or knack for performing her job easily and well.) 

( ) 1. Has great difficulty doing her job. Not at all suited to this kind of 
work. 

( ) 2. Usually has some difficulty doing her job. Not too well suited to this 
kind of work. 

( ) 3. Does her job without too much difficulty. Fairly well suited to this 
kind of work. 

( ) 4. Usually does her job without difficulty. Well suited to this kind of 
work. 

( ) 5. Does her job with great ease. Exceptionally well suited for this kind 
of work. 



C. How large a variety of job duties can she perform efficiently? (Worker *s 
ability to handle several different operations in her work.) 

( ) 1. Cannot perform different operations efficiently. 

( ) 2. Can perform a limited number of different operations efficiently. 

( ) 3. Can perform several different operations with reasonable efficiency. 

( ) 4. Can perform many different operations efficiently. 

( ) 5» Can perform an unusually large variety of different operations 
efficiently. 
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0. How efficiently does worker use her time? (Ability of worker to budget her 

work time*) 

( ) 1, Wastes a considerable amount of work time. Poorly organized work 
habits. 

( ) 2. Work time not used very efficiently. Work habits could stand 
improvement • 

( ) 3« Work time organized fairly efficiently. Work habits acceptable but 
not superior. 

( ) 4. Quite efficient in utilization of her time. Work habits almost 
always superior. 

( ) Makes extremely efficient use of her time. Work habits almost 
always superior. 

E. Considering all the factors already rated, and only these factors, how 

acceptable is her work? (Worker's "all-around" ability to do her job.) 

( ) 1. Would be better off without her. Performance usually not acceptable. 

( ) 2. Of limited value to the organization. Performance somewhat inferior. 

( ) 3. A fairly proficient worker. Performance generally acceptable. 

( ) 4. A valuable worker. Performance usually superior. 

( ) 5« An unusually competent worker. Performance almost always top notch. 
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Appendix B 

Information Recorded for Each Subject 

Age (years) 

Education (years) 

Semesters or quarters of typing training completed 

Time between initial testing and last typing training (months) 

Experience in jobs requiring typing (months) 

Time between initial testing and last typing job 
Information listed under No. 4 or No. 6, whichever is lower 
Initial typing test score — words per minute 
Initial typing test score — errors 
Initial spelling test score (Form T62-M) 

Retest typing test score — words per minute 
Retest typing test score — errors 
Retest spelling test score (T63-M) 

Typing speed change: retest words per minute score minus initial 
typing test words per minute score plus a constant of 100 

Typing accuracy change: retest error score minus initial error score 
plus a constant of 100 

Spelling change: retest spelling test score minus initial spelling test 
score plus a constant of 100 

Hours per day spent typing on job on which followed up 
Experience on job on which followed up (weeks) 

Supervisory rating score (Form SP-21) 

Third Edition D.O.T. six-digit code number 

40 



20 



- 35 - 



Appendix C 

Experimental Design for Follow-up Study on 
USES Typing and Spelling Proficiency Tests 



Purpose: 


To determine the relationships of scores on the USES 
typing and spelling proficiency tests administered to 
applicants before placement with test and job performance 
after placement, and to investigate the factors which 
affect these relationships. 


Allocation 
of Functions: 


Five State agencies will participate in obtaining the 
study data. One of the five will serve as key State 
and will conduct the data analysis. 


Sample: 


The sample is to consist of local office applicants 
tested with the USES typing and spelling proficiency 
tests prior to placement, placed in occupations requiring 
typing, and followed up on the job to obtain retest scores 
and ratings of job performance. Secretary, Stenographer, 
and Typist will be the occupations most frequently repre- 
sented in the sample, but any occupation requiring typing 
skills may be included. 

During the duration of testing for the study in local 
offices, individuals who might ordinarily be referred 
to job openings requiring typing skills without taking 
the typing or spelling tests should be given both of 
these tests. 

Each participating State agency is to submit data for 
a final sample of at least 40 cases. In view of the 
sample attrition which can be expected, it will be 
necessary to initially test considerably more than 40 
individuals. 


Test Administration 
Before Placement: 


Any form of the typing test may be used. It is to be 
administered and scored in accordance with the directions 
in "Directions for Administering and Scoring the New 
USES Typing Test". Individuals may be tested either on 
manual or electric typewriters, but a record should be 
kept as to which type of machine was used by each 
individual . 

41 



Form T62-M of the spelling test is to be given to all 
individuals in the potential sample. It is to be 
administered and scored in accordance with the 
"Directions for Administering and Scoring Spelling 
Test T62-M or T63-M," dated June 1959. 

At the time of testing, the following information is 
to be recorded for each individual in the potential 
sample: 

1. Age (years); 

2. Education (years); 

3. No. of semesters (or quarters) of typing training 
completed; 

4. Length of time elapsed since last typing training 
(months); 

5. Total length of experience in jobs requiring typing 
(months); 

6. Length of time elapsed since last job requiring 
typing (months; record "1" for less than one 
month) . 

To the extent possible (limited only by the unwillingness 
of employers to cooperate) a follow-up should be made for 
each individual in the potential sample who was placed 
by the local office in a job requiring typing skills. 

Between 90 and 120 days after initial testing, (provided 
that the individual has been on the job at least 60 days) , 
the employer should be contacted to ascertain that the 
individual is still employed and in a job requiring typing. 
Arrangements should be made to visit the employer to 
retest the individual as described below and to obtain a 
job performance rating from the individual' s supervisor. 

The visit to the employer should take place within two 
weeks after the initial contact. 

Individuals should be retested on the employer's premises 
on the typewriters which they normally use on their jobs, 
(individuals who were initially tested on manual typewriters 
and are found to be using electric typewriters on the job, 
or vice versa, should be eliminated from the sample before 
retesting.) If a suitable testing situation, free from 
distractions and interruptions, can be achieved at the 
individual's desk she may be tested there, or the testing 
may be done elsewhere on the employer's premises. Any 
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alternate form of the typing test may be used except 
the form used for initial testing of the individual. 

Form T63-M of the spelling test is to be used. Both 
tests are to be administered and scored in accordance 
with the standard directions. 

At the time of retesting, the following information is 
to be recorded for each individual: 



1. Third Edition D.O.T. title and code 

2. Average no. of hours per day spent typing 

3. Length of experience in present job (weeks) 

Job Performance 

Rating: A rating of each individual's job performance is to 

be obtained from her immediate supervisor at approxi- 
mately the same time retesting is done, using Form 
SP-21 (revised 1/66). 

Recording and 

Tabulating of Data: Detailed instructions will be provided by the national 

office at a later date . 

Repoit and 
Submission of 

Data: The participating States should submit their data 

(in a form to be specified by the national office at 
a later date) and a brief report describing the pro^ 
cedures followed to the national office through the 
appropriate regional office by October 31, 1966. 

Data Analysis: All data analysis will be carried out by the key State. 

Detailed instructions will be provided to key State by 
the national office at a later date. 
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Amendment to Experimental Design for Follow-up Study 
on USES Typing and Spelling Proficiency Tests 

Sample: The experimental design specifies that all applicants who are 

given the typing and spelling tests during the initial testing 
phase are to be included in the potential sample for whom 
follow-up data are to be sought. The study objectives will not 
be adequately met, however, if the final sample contains too 
large a proportion of inexperienced applicants. Accordingly, 
the following modification should be made in the composition of 
the final sample of 40 cases: Review the proportion of inexperi- 
enced applicants among those being tested initially (the potential 
sample) and, if necessary, randomly (with no regard to test scores) 
eliminate inexperienced applicants so that those remaining will 
comprise no more than one-fourth of the potential sample for whom 
follow-up data are to be sought. 

Report: The report to be submitted describing the sample and procedures 

followed should include the number of inexperienced applicants so 
eliminated from the potential sample and on the number of other 
individuals in the potential sample who are dropped from the final 
sample for other reasons. 
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Appendix D ! 

I 

Information Recorded for Each Subject ! 

i 

! 

1 . Age (years) 

I 

1 

2. Education (years) ; 

3. Job experience (months) 

4. Typing speed score (wpm.) 

5. Typing accuracy score (errors) 

6. Type of typewriter on which tested (manual or electric) 

7. Third Edition D.O.T. code number 

8. Average number of hours per day spent typing (2-8) 

9. Length of experience on present job (weeks) 

10. Standard Industrial Classification (1967) 

11. Employer's specification, typing speed 

12. Employer's specification, typing accuracy 

i 

] 

f 

! 

13. Supervisor's evaluation, typing speed j 

14. Supervisor's evaluation, typing accuracy 

15. Supervisor's evaluation, overall 
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Experimental Plan for Study on Relationship Between 
Performance on USES Typing Test and Acceptability 
of Typing Performance to Employers 



Purposes: (1) To determine the relationship between scores on the USES 

Typing Test administered to applicants before placement 
and subsequent employers' evaluations of the acceptability 
of their typing performance on the job, and to investigate 
factors which may affect this relationship. 

(2) If warranted by the results in (1) , to establish minimum 
levels of performance on the USES Typing Test which 
correspond to employers' minimum requirements regarding 
typing proficiency. 

Sample: The sample is to consist of individuals who were tested with the 

USES Typing Test prior to placement and were referred to and 
presently hold jobs requiring typing, or who were referred to and 
held such jobs for a significant length of time although they do 
not presently hold them. 

Each of ten participating State agencies is to submit data for a 
final sample of at least 40 cases. Attrition of at least half of 
potential sample members can be anticipated, so it will be 
necessary to contact the employers of considerably more than 
40 individuals in order to attain 40 cases. 

Test Administration: 

Any form of the USES Typing Test may be used. Speed (WPM) 
and accuracy scores are to be derived in accordance with the 
"Directions for Administering and Scoring the New USES Typing 
Test." 

The following additional information should also be obtained for 
each individual in the potential sample at the time of testing: 

1 . Age (years) 

2. Education (years) 

3. Total length of experience at time of testing in jobs 
requiring typing (months) 

4. Type of typewriter (manual or electric) on which tested 
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Individuals may be included in the sample even though data for 
them on length of typing experience are of questionable accuracy. 

Follow-up: To the extent possible (limited only by the unwillingness of 
employers to cooperate) a follow-up attempt should be made 
between 90 and 120 days after testing for each individual in the 
\ potential sample who has been on his job at least 60 days. It 

should first be determined from the employer that the individual 
is still on a job requiring typing or left such a job recently 
! enough to be accurately recalled by his former supervisor, using 

l the same type of typewriter on which he was tested. If these 

\ conditions are met, arrangements should be made to obtain an 

| evaluation of typing performance as described below. The visit 

| to the supervisor should take place as soon as feasible after the 

| initial contact with the employer. 

v 

f Evaluation of Typing Performance: An evaluation of the typing performance of 

| each worker who is still on the job is to be obtained from his 

| immediate supervisor using a sheet similar to the attached 

| specimen "Evaluation Sheet" (Attachment A) . An evaluation 

should also be obtained for each individual no longer on the job 
1 provided that (1) he completed the training period for the particu- 

| lar job before leaving it and (2) he left recently enough that the 

supervisor can clearly recall his typing performance and provide 
a valid evaluation. Supplies of the "Evaluation Sheet" should be 
reproduced by State agencies. The "Evaluation Sheet" also 
provides for the recording of certain other information to be 
obtained from the supervisor regarding the individual. 

The "Evaluation Sheet" is intended to obtain an evaluation of 
typing performance only, independent as far as possible of other 
aspects of job performance. It attempts to identify those indi- 
j viduals whose typing performance is considered marginal or 

inadequate by the supervisor and whom the supervisor would 
like to be able to replace were it not for the difficulties in 
discharging them, the difficulties of obtaining qualified 
replacements, or such other considerations. These points 
should be stressed in discussions with supervisors. The 
complete confidentiality of the evaluations should also be 
strongly emphasized. Evaluations should be made without 
S supervisors' knowledge of Typing Test results. 
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Recording and Tabulating of Data: Data are to be submitted in the form of 
punched cards prepared in accordance with the attached layout 
(Appendix B) . A worktable or tab listing showing these data 
should also be provided. Cards should be submitted only for 
cases with complete data. All punching should be verified. 

Interim Reports: The Quarterly Reports of Test Research Activities for this 
study for July 1 - September 30 and October 1 - December 31 
periods should indicate the total number of workers for whom 
evaluations have been obtained and the number whose overall 
typing performance has been evaluated as unsatisfactory. 

Final Report and Submission of Data: The deck of punched cards and a brief 
report describing the procedures followed should be submitted 
through the appropriate regional office to reach the national office 
by February 28, 1968. The report should indicate the number of 
individuals in the potential sample and the number of these 
eliminated from the final sample for each of the following reasons: 
(1) lack of cooperation by employer; (2) never employed; (3) no 
longer employer, supervisor not able to rate; (4) had not completed 
training period; (5) typing not required on present job; (6) type of 
typewriter used on job (manual or electric) different than type on 
which tested; (7) other (specify) . 

Data Analysis: Data analysis will be carried out by the national office. 



- 43 - 



Appendix F 

(Specimen) Re: 

EVALUATION SHEET 
Instructions to Supervisor 

In an effort to improve our services to you, the 

Employment Service is trying to determine how well the individuals whom 
we refer to typing jobs meet the typing requirements of their employers. 

The person named above was recently referred to your organization by the 

Employment Service. We would now like to 

obtain your evaluation of this person's typing performance. 

This evaluation will be used only for research. It is strictly confidential 
and will not affect the worker in any way. 

A worker who has not completed the training period for the job, who does 
not. spend a significant part of her worktime typing, or who has not been 
on the job or under your supervision long enough for you to know how well 
she types should not be evaluated. Please return the "Evaluation Sheet" 
for any such person with a notation indicating the reason for not making 
the evaluation. 

Your evaluation should be based only on the worker* s typing performance . 

In making the evaluation, do not consider her conduct or how she performs 
the other duties of her job, only her typing. Make your evaluation in terms 
of her usual or typical performance ; not on the basis of one "good" day, 
one "bad" day, or some single incident. 

Evaluation 

Check the statement below which best describes the worker's speed of 
typing, accuracy of typing, and overall typing performance (three checks). 
Remember to consider only her typing performance. 



Speed 


Accuracy 


Overall 


Evaluation 








Fully satisfactory 








Unsatisfactory or 
only barely 
satisfactory 



Worker* s Job Title 

Length of Experience on Present Job: weeks 

Average Number of Hours per Day Spent Typing: 

Kind of Typewriter Used by Worker: Manual 
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Code Numbers and Job Titles from the Third Edition of the Dictionary of 
Occupational Titles 





Code Number 


Job Title 


1. 


132.268 


Reporter, Correspondent, Feature Reporter, 
Rewrite Man 


2. 


169.168 


Adjudicator, Administrative Assistant, 
Administrative Secretary, Appeal Referee, 
Chief Clerk, Civil Defense Training 
Officer, Estate Planner, Examiner, Fire 
Assistant, Interviewer and Claims Adjuster, 
Laboratory Assistant, Manager, Park Ranger, 
Property Disposal Officer, Occupational 
Rating Specialist, Vital Statistics Registrar, 
Special Agent, Vocational Disability 
Examiner 


3. 


201.368 


Secretary, Legal Secretary, Medical Secretary 


4. 


202.388 


Court Reporter, Stenographer, Stenotype 
Operator 


5. 


203.138 


Chief Telegraphic - Typewriter Operator, 
Typing Section Chief 


6. 


203.588 


Telegraphic - Typewriter Operator, Typist, 
Perforator Typist, Wires - Transfer Clerk 


7. 


208.588 


Braille - Machine Operator, Cryptopgraphic - 
Machine Operator, Photocomposing - Machine 
Perforator Operator, Telautograph Operator, 
Transcribing - Machine Operator, Typesetter - 
Perforator Operator 


8. 


209.382 


Justowriter Operator 


9. 


209.388 


Cancellation Clerk, Classified - Ad Clerk, 
Clerk - Typist, Continuity Clerk, Mortgage 
Clerk, Mortgage Processing Clerk, Reader, 
Reference Clerk, Special - Certificate 
Dictator, Statement Clerk, Title Searcher 

50 
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Code Number 


Job Title 


10. 


209.588 


Addresser, Car Checker, General Clerk, 

Clerk - Typist, Bardereau Clerk, Copy Holder, 
Credit Card Clerk, Customer Control Clerk, 
History Card Clerk, Library Clerk, Marker, 
Price Clerk, Return to Factory Clerk, Routing 
Clerk, Translator - Braille 


11. 


210.388 


Audit Clerk, Distribution Accounting Clerk, 
Reconcilement Clerk, Billing Control Clerk, 
Bookkeeper, Night Auditor, Account Classi- 
fication Clerk, Classification Control Clerk, 
Commodity Loan Clerk, Fixed - Capital 
Clerk, Medical Voucher Clerk, Mortgage 
Loan Computation Clerk 


12. 


211.368 


Cashier I 


13. 


211.468 


Cashier II 


14. 


215.388 


Bookkeeping - Machine Operator 


15. 


219.388 


Account Analyst , Actuarial Clerk, Aircraft - 
Log Clerk, Billing Clerk, Bond Clerk, Box - 
Estimator, Budget Clerk, Checker, City 
Collection Clerk, Clearing-House Clerk, 
Cable Transfer Clerk, General Office Clerk, 
Telegraph Service Clerk, Statistical Clerk, 
Timekeeper, etc. 


16. 


219.488 


Accounting Clerk, Advertising - Space 
Measurer, Booking Prizer, Brokerage Clerk, 
Collection Sheet Clerk, Cost Clerk, Deposit 
Refund Clerk, Fabric and Accessories 
Estimator, Foreign Clerk, Grading Clerk, 
Insurance Checker, Rate Clerk, Toll - Rating 
Clerk 


17. 


219.588 


Coin - Machine Operator, Posting Clerk, 
Bookman, Charge - Refer Clerk, Data - 
Change Clerk, Error - Ledger Clerk, Kardex 
Clerk, Meter - Order Clerk, Remittance 
Clerk, Suspense Clerk, Telegraph Service 
Rater 
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Code Number 


Job Title 


18. 


221.388 


Chart Clerk, Expediter Clerk, Floor Clerk, 

Gas - Plant Clerk, Industrial - Order Break - 
Down Clerk, Manufacturing Order Clerk, 

Mill Recorder, Order Detailer, Formula 
Checker, Provider, Overhaul Planner, 
Production Clerk, Recorder, Traffic Man, 
Weight and Test Bar Clerk, Work - Order Clerk 


19. 


237.368 


Admitting Officer, Appointment Clerk, 
Information Clerk , Out- Patient Admitting 
Clerk, Receptionist, Registrar, Registration 
Clerk 


to 

o 

• 


249.368 


Claims Clerk, Electoral Clerk, Counter Clerk, 
Court Clerk, Credit Clerk, Credit Reporter, 
Deposit Clerk, Form Checker, Fingerprint 
Clerk, Library Assistant, License Clerk, 

Loan Officer, New - Account Clerk, Order 
Clerk, Reconsignment Clerk, Religious - 
Affairs Clerk, Reservation Clerk, Safe - 
Deposit Clerk, Test Technician, Tracer 
Clerk 


21. 


289.458 


Salesperson 


CO 

CO 

• 


299.388 


Stamp Classifier 


CO 

CO 

• 


All other 
numbers 


All code numbers with an "X" are listed in 
the category "all other numbers". An 'X" 
is used to denote an entry worker who is not 
fully qualified to carry out the duties of a 
specific occupation because of lack of 
sufficient training or experience. 
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